Image processing apparatus, control method thereof, and storage medium

ABSTRACT

An image processing apparatus obtains an image captured in an image capturing and sets a white balance recording mode for recording the image as a still image and a white balance display mode for displaying the image before being captured as the still image on a display. The apparatus controls white balance correction performed on the image captured in the image capturing. In a case where a first display mode that is included in the display mode is set, the apparatus sets an amount of a light source color that remains after the white balance correction is performed on the image before being captured as the still image in the first display mode to be larger than the amount of the light source color that remains after the white balance correction is performed on the still image in the recording mode.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image processing apparatus, acontrol method thereof, and a storage medium.

Description of the Related Art

Conventionally, a mirrorless camera that includes an electronicviewfinder (EVF) instead of an optical finder is known. With thiscamera, image processing parameters that are the same as those appliedto a still image that is recorded is applied to an image that isdisplayed on the EVF, and thus a photographer can check how a finishedstill image will look like before capturing the image.

On the other hand, when image processing parameters that are the same asthose applied to a still image that is recorded is applied to an imagethat is displayed on the EVF, the color tone may be different from thecolor tone of what the photographer actually sees with the naked eye inthe image capturing environment. For this reason, a photographer who isaccustomed to capturing images using a camera with an optical finder mayfeel uncomfortable when looking at the images displayed on the EVF.Specifically, a recorded image may appear more favorable by, when imagecapturing is performed using an auto white balance function, correctingthe reddish color of a light bulb color light source, the bluish colorof a shade, and the like such that the atmosphere of a light sourcecolor does not remain as compared to that seen with the naked eye, andthen recording the image as a still image. Accordingly, when whitebalance correction values that are applied to still images that recordedare applied to the images displayed on the EVF, a significant differencemay occur between how the images displayed on the EVF look like and howa subject looks like with the naked eye.

Japanese Patent Laid-Open No. 2002-218495 is known as a document thatproposes a technique for recording still images using an auto whitebalance function. Specifically, Japanese Patent Laid-Open No.2002-218495 proposes a technique in which, when a photographer recordsstill images using an auto white balance function, the photographermanually sets the degree of correction in auto white balance applied tothe still images, and sets the degree of the atmosphere of a lightsource color to a desired degree.

In the case where the technique disclosed in Japanese Patent Laid-OpenNo. 2002-218495 is used together with displaying on the EVF so as tobring the color tone of images displayed on the EVF close to the colortone of what looks like with the naked eye (the degree of the atmosphereof a light source color is increased), the light source color may remainconsiderably in not only the images that are displayed but also thestill images that are recorded. That is, a technique is desired thatreduces the color tone of a light source in a still image that isrecorded, and also controls the color tone of the light source in imagesthat are displayed on the EVF.

SUMMARY OF THE INVENTION

The present disclosure has been made in consideration of theaforementioned issues, and realizes a technique with which, even whenthe color tone of a light source in a still image that is recorded isreduced, it is possible to bring the color tone of the light source inan image that is displayed on the EVF close to the color tone of whatlooks like with the naked eyes.

In order to solve the aforementioned problems, one aspect of the presentdisclosure provides an image processing apparatus comprising: one ormore processors; and a memory storing instructions which, when theinstructions are executed by the one or more processors, cause the imageprocessing apparatus to function as: an obtaining unit configured toobtain an image captured by an image capturing unit; a setting unitconfigured to be capable of setting a white balance recording mode thatis applied to record the image as a still image and a white balancedisplay mode that is applied to display the image before being capturedas the still image on a display unit; and a control unit configured tocontrol white balance correction performed on the image captured by theimage capturing unit, wherein, in a case where a first display mode thatis included in the display mode is set, the control unit sets an amountof a light source color that remains after the white balance correctionis performed on the image before being captured as the still image inthe first display mode to be larger than the amount of the light sourcecolor that remains after the white balance correction is performed onthe still image in the recording mode.

Another aspect of the present disclosure provides, a control method ofan image processing apparatus comprising: obtaining an image captured byan image capturing unit; setting a white balance recording mode that isapplied to record the image as a still image and a white balance displaymode that is applied to display the image before being captured as thestill image on a display unit; and controlling white balance correctionperformed on the image captured by the image capturing unit, wherein, inthe controlling, in a case where a first display mode that is includedin the display mode is set, an amount of a light source color thatremains after the white balance correction is performed on the imagebefore being captured as the still image in the first display mode isset to be larger than the amount of the light source color that remainsafter the white balance correction is performed on the still image inthe recording mode.

Still another aspect of the present disclosure provides a non-transitorycomputer-readable storage medium comprising instructions for performinga control method of an image processing apparatus, the control methodcomprising: obtaining an image captured by an image capturing unit;setting a white balance recording mode that is applied to record theimage as a still image and a white balance display mode that is appliedto display the image before being captured as the still image on adisplay unit; and controlling white balance correction performed on theimage captured by the image capturing unit, wherein, in the controlling,in a case where a first display mode that is included in the displaymode is set, an amount of a light source color that remains after thewhite balance correction is performed on the image before being capturedas the still image in the first display mode is set to be larger thanthe amount of the light source color that remains after the whitebalance correction is performed on the still image in the recordingmode.

According to the present invention, even when the color tone of a lightsource in a still image that is recorded is reduced, it is possible tobring the color tone of the light source in an image that is displayedon the EVF close to the color tone of what looks like with the nakedeye.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an example of a functionalconfiguration of a digital camera as an example of an image processingapparatus according to a first embodiment.

FIG. 2 is a block diagram showing an example of a functionalconfiguration of a white balance control unit according to the firstembodiment.

FIG. 3 is a flowchart illustrating a series of operations of imagecapturing processing according to the first embodiment.

FIG. 4 is a flowchart illustrating a series of operations of whitebalance correction value calculation processing according to the firstembodiment.

FIG. 5 is a diagram illustrating a white detection range according tothe first embodiment.

FIGS. 6A to 6D are diagrams illustrating limiter control framesaccording to the first embodiment.

FIGS. 7A to 7D are diagrams illustrating control of the limiter controlframes according to the first embodiment.

FIG. 8 is a diagram illustrating a relationship between the remainingamount of the light source color and combinations of white balance modesaccording to the first embodiment.

FIG. 9 is a block diagram showing an example of a functionalconfiguration of a white balance control unit according to a secondembodiment.

FIG. 10 is a flowchart illustrating a series of operations of whitebalance correction value calculation processing according to the secondembodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the claimed invention. Multiple features aredescribed in the embodiments, but limitation is not made to an inventionthat requires all such features, and multiple such features may becombined as appropriate. Furthermore, in the attached drawings, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

Hereinafter, an example will be described in which a digital camera thatcan capture still images as an example of an image processing apparatus.However, the present embodiment is not limited to a digital camera thatcan capture still images, and is also applicable to a digital camerathat can capture moving images, and any of electronic devices such as amobile phone and a computer that can capture images.

First Embodiment

Configuration of Digital Camera

FIG. 1 is a block diagram showing an example of a functionalconfiguration of a digital camera as an example of an image processingapparatus according to the present embodiment. One or more functionalblocks shown in FIG. 1 may be implemented using hardware such as an ASICor a programmable logic array (PLA), or by a programmable processor suchas a CPU or a GPU executing software. Alternatively, one or morefunctional blocks shown in FIG. 1 may be implemented by a combination ofsoftware and hardware. The processing operations described below may beperformed mainly by the same hardware even when it is described that theprocessing operations are performed by different functional blocks.

An optical system 101 includes, for example, a lens, a shutter, and anaperture, and exposes an imaging element 102 with an optical image of asubject. The optical system 101 is configured to be capable ofperforming communication with a central processing unit (hereinafter,referred to simply as “CPU”) 103, and transmits information regardingfocal distance, the shutter speed, the aperture value, and the like tothe CPU 103.

The imaging element 102 may be, for example, a CCD image sensor, a CMOSimage sensor, or the like, and functions as an image capturing unit. Theimaging element 102 includes, for example, an RGB Bayer pattern or thelike, and converts an optical image imaged by the optical system 101into luminance information on a pixel by pixel basis. The luminanceinformation is digitized through an AD converter (not shown), and storedin a primary storage device 104 as RAW data before being subjected todevelopment processing. The electrical gain (hereinafter, referred tosimply as “ISO sensitivity”) of the imaging element 102 is set by theCPU 103.

A photometer device 105 divides the RAW data into a plurality ofphotometric regions, and detects a subject luminance for each regionfrom the amount of incident light by the optical system 101. Thephotometer device 105 transmits data regarding the detected subjectluminance to the CPU 103.

The CPU 103 functions as a control unit of a digital camera 100. The CPU103 controls the constituent units that constitute the digital camera100 based on signals input from other structural elements and programsstored in advance in a secondary storage device 108 or the like, so asto implement various functions of the digital camera 100.

The primary storage device 104 is, for example, a volatile storagemedium such as a RAM, and is used as a work memory by the CPU 103. Also,the information stored in the primary storage device 104 may be used byan image processing apparatus 106, or may be recorded in a storagemedium 107.

The secondary storage device 108 is, for example, a nonvolatile storagedevice such as an EEPROM, and stores a program (firmware or anapplication) for controlling the digital camera 100 and various types ofsettings information. The program stored in the secondary storage device108 is read and executed by the CPU 103. Also, the settings informationstored in the secondary storage device 108 is also read and used by theCPU 103. Various setting values for image processing used in the presentembodiment, which will be described later, are also stored in thesecondary storage device 108.

The storage medium 107 is, for example, a nonvolatile storage mediumsuch as a semiconductor memory, and records image data and the like thatare obtained by image capturing and stored in the primary storage device104. The storage medium 107 may be configured to be capable of beingattached and detached to and from the digital camera 100 such as, forexample, a semiconductor memory card. In this case, the data recorded inthe storage medium 107 can be read by other devices such as a personalcomputer. As described above, the digital camera 100 may have at leastone of the attaching/detaching mechanism and the reading/writingfunction of the storage medium 107.

A display unit 109 functions as display unit, and displays viewfinderimages for capturing images, captured images, GUI images for interactionoperations, and the like. The display unit 109 may include a pluralityof display panels. For example, viewfinder images and captured imagesmay be displayed on different display panels.

An operation unit 110 is a group of input devices that receives useroperations and transmits input information to the CPU 103, and mayinclude, for example, a button, a lever, a touch panel, and inputdevices that use voice, line of sight, and the like. The operation unit110 also includes a release button for starting image capturing. Thedigital camera 100 of the present embodiment has a plurality of imageprocessing patterns that are applied to captured images by the imageprocessing apparatus 106, and any one of the patterns can be set as animage capturing mode from the operation unit 110.

The image processing apparatus 106 performs, on the RAW data beforebeing subjected to development processing, image processing that is alsoreferred to as “development processing” such as white balanceprocessing, color interpolation processing for converting an RGB Bayerpattern signal to three RGB plain signals, gamma correction processing,chroma correction, hue correction, and the like. White balancecomputation performed in the present embodiment is also performed by theimage processing apparatus 106. At least a portion of the functions ofthe image processing apparatus 106 may be implemented by the CPU 103executing a program.

Configuration of Image Processing Apparatus

Next, an example of a configuration of a white balance control unit 200that performs white balance processing will be described with referenceto FIG. 2 . The white balance control unit 200 shown in FIG. 2 isincluded in, for example, the image processing apparatus 106. Thedigital camera 100 of the present embodiment implements an auto whitebalance function of automatically computing the white balance accordingto the light source in a photographic scene by including the whitebalance control unit 200.

The white balance control unit 200 may be a circuit or a software modulethat performs auto white balance computation. The white balance controlunit 200 includes a block dividing unit 201, a white determining unit202, a white balance correction value limiting unit 203, and a whitebalance correction value calculation unit 204. Details of the processingperformed in each block will be described later.

The digital camera 100 of the present embodiment has the following twodisplay modes as operation modes for displaying images before beingcaptured as still images on the display unit 109 as viewfinder images.One of the display modes is a mode (referred to as “still imagerecording equivalent display mode”) for performing exposure control andcontrolling image processing operations such as white balance, gammacorrection, and color correction processing, with which the viewfinderimages have a finish similar to that of that captured still images. Theother display mode is a mode (referred to as “naked eye view equivalentdisplay mode”) for performing exposure control and controlling imageprocessing operations such as white balance, gamma correction, and colorcorrection processing, with which the viewfinder images have a finishthat is different from that of the captured still images and is close towhat looks like with the naked eye. In the two display modes, differentauto white balance computations are performed. Details of the twodisplay modes will be described later.

Furthermore, the digital camera 100 also has, in addition to the twodisplay modes described above, two still image recording modes as autowhite balance modes that are applied to images that are recorded asstill images. One of the still image recording modes is a mode (referredto as “white priority mode”) for performing control so as to reduce thedegree to which the atmosphere of the light source color is left in autowhite balance applied to still images (also referred to as “theremaining amount of the light source color”). The other still imagerecording mode is a mode (referred to as “atmosphere priority mode”) inwhich the remaining amount of the light source color applied to stillimages is set to be larger than that of the white priority mode.

In the case where the still image recording equivalent display mode isset, the image processing apparatus 106 performs, on images that aredisplayed on the display unit 109, auto white balance computationsimilar to that of the recording mode (the white priority mode or theatmosphere priority mode) that has been set. As a result, in the stillimage recording equivalent display mode, the digital camera 100 canoutput images (viewfinder images) with a color tone similar to that ofstill images that are output to the display unit 109 in the whitepriority mode or the atmosphere priority mode.

On the other hand, in the case where the naked eye view equivalentdisplay mode is set, the image processing apparatus 106 performs autowhite balance computation that is independent of the recording mode (thewhite priority mode or the atmosphere priority mode) that has been set.For example, the image processing apparatus 106 performs, on the imagesthat are displayed on the display unit 109, auto white balancecomputation for leaving the light source color in an amount more thanthat in the case of the atmosphere priority mode, irrespective of therecording mode. As a result, the digital camera 100 can output, to thedisplay unit 109, images with a color tone close to that of the ambientlight source color that is viewed by the user (or in other words, thephotographer) with the naked eye.

Series of Operations of Image Recording by Image Capturing

Next, a series of operations of image recording by image capturing willbe described with reference to FIG. 3 . The processing operations areimplemented by the CPU 103 deploying a program stored in the secondarystorage device 108 into the primary storage device 104, executing theprogram, and causing the imaging element 102, the image processingapparatus 106, and the like (including internal structural elements) tooperate.

In S301, the CPU 103 receives a user input regarding image capturingfrom the operation unit 110. The user input includes, for example,inputs regarding various settings of the digital camera 100 such as thefocal distance of the optical system 101, the shutter speed, theaperture value, the ISO sensitivity, the exposure correction amountsetting, the manual exposure, and the auto exposure. Also, the displaymode (the still image recording equivalent display mode or the naked eyeview equivalent display mode) and the recording mode (the white prioritymode or the atmosphere priority mode) are also set.

In S302, the CPU 103 performs photometric control for image capturing.For example, in the case where an auto exposure mode is set as a settingfor image capturing in S301, the CPU 103 divides an image output fromthe imaging element 102 into n by m regions, where n represents thenumber of regions in the horizontal direction, and m represents thenumber of regions in the vertical direction. n and m may be anyintegers. In the present embodiment, for example, both n and m are 16.The CPU 103 divides the image into a total of 256 regions, and obtainsan RGB signal from each of divided regions. Then, the CPU 103 calculatesY values from the RGB signals from the regions based on Equation 1, andfurther calculates the average value of the Y values of the regions tocalculate the brightness of the subject.Y=3×R+6×G+B  Equation 1

By calculating the brightness of the subject, the CPU 103 can controlthe ISO sensitivity, the shutter speed, and the aperture value based onthe calculated brightness of the subject such that the image displayedon the display unit 109 has a brightness designated by the user.

In S303, the CPU 103 adjusts the settings for image capturing. Forexample, the CPU 103 adjusts the settings such as the focal distance ofthe optical system 101, the shutter speed, the aperture value, and theISO sensitivity of the imaging element 102 based on the settingsinformation received in S301 and the brightness of the subjectcalculated in S302. In the case where a manual exposure mode is set inS301, exposure is controlled based on the ISO sensitivity, the shutterspeed, and the aperture value that were designated by the user. However,in the case where the display mode is set to the naked eye viewequivalent display mode, the CPU 103 operates such that the imagedisplayed on the display unit 109 is in the auto exposure mode even whenthe manual exposure mode has been set. At this time, for exposure at thetime of still image recording in S310, which will be described later,the CPU 103 performs image capturing based on the settings designated bythe user as manual exposure settings.

In S304, the CPU 103 controls the imaging element 102 for imagecapturing. For example, the CPU 103 further adjusts the settings (ISOsensitivity and the like) of the imaging element 102 based on thesettings information for image capturing adjusted in S303, and performsimage capturing processing. In S305, the CPU 103 performs focus control.For example, the CPU 103 performs auto focus control based on thecaptured image so as to focus at a focus position at which the contrastis highest. The auto focus control can be performed using a knownmethod. The above-described order in which S302 to S305 are performed ismerely an example, and thus the present embodiment is not limited tothis order. The order in which S302 to S305 are performed may be changedaccording to the processing.

In S306, the CPU 103 calculates white balance correction values for thecaptured image using the image processing apparatus 106. Details of thewhite balance correction value calculation will be given later after aflow of a series of image capturing operations performed until S314 isdescribed.

In S307, the CPU 103 performs development processing. For example, theCPU 103 controls the image processing apparatus 106 to performdevelopment processing operations such as white balance correctionprocessing using the white balance correction value calculated in S306,color interpolation processing, gamma correction processing, chromacorrection processing, and hue correction processing. Then, in S308, theCPU 103 displays the image developed in S307 on the display unit 109.

In S309, the CPU 103 determines whether a user instruction to startstill image capturing has been received by the operation unit 110. Forexample, if it is determined based on the user input received by theoperation unit 110 that the instruction to start still image capturinghas not been received, the CPU 103 returns the processing to S302, andrepeats the operations from S302 to S308. On the other hand, if it isdetermined based on the user input received by the operation unit 110that the instruction to start still image capturing has been received,the CPU 103 advances the processing to S310, and transitions into stillimage capturing control.

In S310, the CPU 103 captures a still image based on the settings forstill image capturing such as the ISO sensitivity, the shutter speed,and the aperture value. In the case where the manual exposure mode isset, the CPU 103 may set the ISO sensitivity, the shutter speed, and theaperture value to be the same as the setting values calculated in S302.Also, in the case where the auto exposure mode is set, the CPU 103 mayset the ISO sensitivity and the like to be the same as the settingvalues calculated in S302, or may separately calculate the settings forstill image capturing when calculation is performed in the photometriccontrol in S302 and use the calculated setting values. In S311, the CPU103 stores the image captured in S310 in the primary storage device 104.

In S312, the CPU 103 calculates white balance correction values for theimage stored in the primary storage device 104 in S311. In the whitebalance correction value calculation performed here, white balancecorrection values that correspond to either one of the white prioritymode and the atmosphere priority mode of the recording mode that hasbeen set are calculated. Details thereof will be described later.

In S313, the CPU 103 performs development processing. For example, theCPU 103 controls the image processing apparatus 106 to performdevelopment processing operations such as white balance correctionprocessing that uses the white balance correction value calculated inS312, color interpolation processing, gamma correction processing,chroma correction processing, and hue correction processing. Then, inS314, the CPU 103 records the image developed in S313 in the storagemedium 107. After that, the CPU 103 ends the series of operations.

White Balance Correction Value Calculation Processing

Next, a detailed description of the white balance correction valuecalculation processing (in S306 and S312) described above will be givenwith reference to FIG. 4 . The processing is implemented by the CPU 103executing a program stored in the secondary storage device 108, or bythe CPU 103 controlling the image processing apparatus 106.

In S401, the CPU 103 divides the captured image into block regions usingthe block dividing unit 201 of the image processing apparatus 106. Forexample, the CPU 103 divides, using the block dividing unit 201, thecaptured image into n by m regions, where n represents the number ofregions in the horizontal direction, and m represents the number ofregions in the vertical direction. n and m may be any integers. In thepresent embodiment, for example, both n and m are 16. That is, the blockdividing unit 201 divides the image into a total of 256 regions. In thefollowing description, the divided regions will be referred to as “blockdivided regions”.

In S402, the CPU 103 obtains Rave, Gave, and Bave that represent theaverage values of R, G, and B signals in each block divided region basedon the following Equation 2.Rave=Rall/RcountGave=Gall/GcountBave=Ball/Bcount  Equation 2

Here, Rall, Gall, and Ball represent the integrated values of R, G, andB signals in each block divided region. Rcount, Gcount, and Bcountrespectively represent the number of R signals, the number of G signals,and the number of B signals in each block divided region.

In S403, the CPU 103 calculates the R and B signals normalized by the Gsignals (R/G value and B/G value) in each block divided region based onEquation 3.R/G=Rave/GaveB/G=Bave/Gave  Equation 3

In S404, the CPU 103 performs white determining processing using thewhite determining unit 202 of the image processing apparatus 106.Specifically, the white determining unit 202 extracts blocks plottedwithin a white detection range 907 shown in FIG. 5 for the R/G and B/Gvalues in each block region. Here, the white detection range 907 will bedescribed in detail with reference to FIG. 5 .

Reference numeral 501 shown in FIG. 5 indicates a plotted position of alight source on a black body radiation trajectory (also referred to as“black body emission trajectory”) in a graph with the x axisrepresenting R/G and they axis representing B/G. Also, referencenumerals 502 to 507 schematically indicate examples of plotted positionsof various types of light sources in the coordinate system shown in FIG.5 . Reference numeral 502 indicates an example of a plotted position ofsunlight (about 5200 kelvin (hereinafter, simply indicated by K)),reference numeral 503 indicates an example of a plotted position ofshade (about 7000 K), reference numeral 504 indicates an example of aplotted position of a light bulb color light source (about 2800 K),reference numeral 505 indicates an example of a plotted position of awhite fluorescent light, reference numeral 506 indicates an example of aplotted position of a daylight white fluorescent light, and referencenumeral 507 indicates an example of a plotted position of a daylightcolor fluorescent light. The white detection range 907 is set so as toinclude the light sources on the black body radiation trajectory and thefluorescent light sources that are indicated by reference numerals 502to 507. As a result of the white detection range 907 being set in themanner described above, by using a known auto white balance method, itis possible to extract pixels that may have light source colors in theimage region and correct white balance by changing the pixel values ofthe extracted pixels to achromatic color.

Next, the white determining unit 202 calculates, for each block plottedin the white detection range 907, RaveAll, GaveAll, and BaveALL thatrepresent the average values of Rave, Gave, and Bave that represent theaverage values of R, G, and B signals in the block based on Equation 4.RaveALL=(Rave integrated value of extracted blocks)/(the number ofextracted blocks)GaveALL=(Gave integrated value of extracted blocks)/(the number ofextracted blocks)BaveALL=(Bave integrated value of extracted blocks)/(the number ofextracted blocks)  Equation 4

If no block is plotted within the white detection range 907, the whitedetermining unit 202 calculates the average values RaveAll, GaveAll, andBaveALL using Rave, Gave, and Bave that represent the average values ofR, G, and B signals in all blocks. Then, the white determining unit 202calculates R/Gall and B/Gall, which are the R and B signals normalizedby the G signals, based on Equation 5.R/Gall=RaveALL/GaveALLB/Gall=BaveALL/GaveALL  Equation 5

Through the processing up to here, a pair of R and B signals normalizedby G signals (or in other words, one point within a plane based on B/Gand R/G) is calculated. The white determining unit 202 ends the whitedetermining processing in S402.

In the present embodiment, as described above, an example of a methodfor extracting pixels that may have light source colors from an imagehas been described. However, the method for determining the valuescorresponding to R/Gall and B/Gall is not limited thereto, and any othermethod may be used. For example, a method may be used in which thevalues corresponding to R/Gall and B/Gall are obtained directly from animage by using deep learning that is an example of machine learning. Forexample, a deep neural network may be trained using pairs of images andvalues of R/Gall and B/Gall that correspond to the images as teacherdata. Then, by inputting an image to the trained deep neural network,estimated values of R/Gall and B/Gall can be obtained.

In S405, the CPU 103 performs limiter processing using the white balancecorrection value limiting unit 203 of the image processing apparatus106. The limiter processing in S405 is processing for limiting theamount of correction of the white balance correction value in order toleave the light source color as white balance for the pair of R/Gall andB/Gall calculated in S404. Hereinafter, the limiter processing will bedescribed in detail with reference to FIGS. 6A to 6D.

FIG. 6A shows an example of a limiter control frame 608 used in thelimiter processing. Reference numeral 601 shown in FIG. 6A indicates aplotted position of a light source on a black body radiation trajectoryin a graph with the x axis representing R/G and the y axis representingB/G, as with reference numeral 501 shown in FIG. 5 . Also, referencenumerals 602 to 607 indicate plotted positions of light sources on theblack body radiation trajectory and fluorescent light sources (sunlight,shade, a light bulb color light source, a white fluorescent light, adaylight white fluorescent light, and a daylight color fluorescent lightin this order), which are indicated by reference numerals 502 to 507 inFIG. 5 .

In the case where R/Gall and B/Gall calculated in S404 are plottedoutside the limiter control frame 608, R/Gall and B/Gall are moved topositions on the closest limiter control frame. Then, R/Gall_Limit andB/Gall_Limit, which will be R/Gall and B/Gall after the limiterprocessing, are calculated. In the example shown in FIG. 6A, referencenumerals 603 to 607 are plotted outside the limiter control frame 608.Accordingly, in the case where R/Gall and B/Gall calculated in S404 areplotted at the same position as any of the plotted positions 603 to 607,R/Gall_Limit and B/Gall_Limit are calculated. On the other hand,reference numeral 602 is plotted within the limiter control frame 608,and thus in the case where R/Gall and B/Gall are plotted at the sameposition as reference numeral 602, it is unnecessary to perform themoving processing described above.

When the limiter processing ends, the CPU 103 calculates, using thewhite balance correction value calculation unit 204, R gain, G gain, andB gain that represent the final white balance correction values (gainsapplied to R, G, and B signals) based on Equation 6. As a result,through the white balance correction, for the light source 602, thelight source color does not remain, and for each of the light sources603 to 607, the light source color remains.Rgain=1/(R/Gall_Limit)Ggain=1Bgain=1/(B/Gall_Limit)  Equation 6

Here, a description will be given of processing for leaving the lightsource color for the auto white balance mode in the display mode (thestill image recording equivalent display mode or the naked eye viewequivalent display mode) and the recording mode (the white priority modeor the atmosphere priority mode) according to the present embodiment.The limiter control frame 608 is set such that the size of the areavaries according to the brightness. That is, the limiter control frame608 is set to have a smaller area as the brightness is higher, and isset to have a larger area as the brightness is lower. The brightnessused here is the brightness of the subject calculated in photometriccontrol in S302. The reason that the limiter control frame is set tohave a smaller area as the brightness is higher is as follows. When thebrightness is high, it is highly likely that the light source is outdoorsunlight. For this reason, it is unnecessary to perform white balancecorrection for light sources other than sunlight. Also, it is possibleto reduce erroneous corrections and overcorrections of white balancethat are caused as a result of the color of the subject beingerroneously detected as the light source color in the above-describedwhite determining processing.

The limiter control frame 608 shown in FIG. 6A was given as an exampleof a limiter control frame when the brightness is high in the whitepriority mode, whereas FIG. 6B shows an example of a limiter controlframe 609 when the brightness is low in the white priority mode. Whenthe brightness is low, the light sources 602 to 607 are set to beplotted within the limiter control frame 609. As a result, for each ofthe light sources 602 to 607, the light source color does not remainafter the white balance correction. The limiter control frame when thebrightness is high may be used in common in the auto white balance modein the display mode (the still image recording equivalent display modeand the naked eye view equivalent display mode) and the recording mode(the white priority mode and the atmosphere priority mode). On the otherhand, when the brightness is low, the limiter control frame is changedin each mode. That is, in the present embodiment, if the brightness isgreater than or equal to a brightness threshold value, the colortemperature (or in other words, the amount of the light source colorthat remains after the white balance correction) of the limiter controlframe is set to be the same between display modes. On the other hand,when the brightness is lower than the brightness threshold value, thecolor temperature (the amount of the light source color that remainsafter the white balance correction) of the limiter control frame ischanged between modes. At this time, the brightness threshold value isset to a brightness at which, for example, the color temperature is 5000K in FIGS. 7A to 7D, which will be described below.

FIG. 7A shows an example of control on the low color temperature side ofthe limiter control frame according to the brightness in the whitepriority mode of the recording mode. In FIGS. 7A to 7D, the x axisrepresents brightness, and they axis represents the plotted position ofthe color temperature on the low color temperature side of the limitercontrol frame. FIG. 7A shows an example in which, when the brightness ofthe subject is high, the color temperature of the limiter control frameis set to 5000 K, and the color temperature is gradually reduced to 2800K as the brightness decreases. 5000 K shown in FIG. 7A is a colortemperature that corresponds to the low color temperature side of thelimiter control frame 608 when the brightness is high, which was shownin FIG. 6A. Also, 2800 K shown in FIG. 7A is a color temperature thatcorresponds to the low color temperature side of the limiter controlframe 609 when the brightness is low, which was shown in FIG. 6B. Thehigh color temperature side of the limiter control frame and thefluorescent light side relative to the black body radiation trajectory(on the origin side relative to the black body radiation trajectory) canalso be controlled such that the color temperature is gradually changedaccording to the brightness, as with the control on the low colortemperature side of the limiter control frame.

FIG. 6C shows an example of a limiter control frame 610 when thebrightness is low in the atmosphere priority mode of the recording mode.When the brightness is low in the atmosphere priority mode, the limitercontrol frame 610 is set such that the light sources 603 to 607 arelocated at positions slightly outside the limiter control frame 610. Asa result, for each of the light sources 603 to 607, a small amount ofthe light source color remains after the white balance correction.

FIG. 7B is a diagram showing control on the low color temperature sideof the limiter control frame according to the brightness in theatmosphere priority mode. FIG. 7B shows an example in which, when thebrightness is high, the color temperature is set to 5000 K, and thecolor temperature is gradually reduced to 3200 K as the brightnessdecreases. 5000 K shown in FIG. 7B is a color temperature thatcorresponds to the low color temperature side of the limiter controlframe 608 when the brightness is high, which was shown in FIG. 6A. Also,3200 K shown in FIG. 7B is a color temperature that corresponds to thelow color temperature side of the limiter control frame 610 when thebrightness is low, which was shown in FIG. 6C.

As described above, in the case where the still image recordingequivalent display mode of the display mode is selected, the CPU 103sets a limiter control frame that corresponds to either one of the whitepriority mode and the atmosphere priority mode of the recording modethat has been set, and computes white balance correction values. Thatis, in the case where the still image recording equivalent display modeis set as the display mode, the amount of the light source color thatremains after the white balance correction has been performed on theimage is set to be equal to the amount of the light source color thatremains after the white balance correction has been performed on thestill image in the recording mode.

On the other hand, in the case where the naked eye view equivalentdisplay mode of the display mode is selected, the CPU 103 sets a limitercontrol frame that is different from the limiter control frame that isset in the recording mode (the white priority mode and the atmospherepriority mode).

FIG. 6D shows a limiter control frame 611 when the brightness is low inthe naked eye view equivalent display mode. When the brightness is lowin the naked eye view equivalent display mode, the limiter control frame611 is set to have a smaller area than the limiter control frame 610that is set when the brightness is low in the atmosphere priority mode.As a result, the light sources 603 to 607 are significantly spaced apartfrom the limiter control frame, and for each of the light sources 603 to607, a large amount of the light source color remains after the whitebalance correction. As described above, the limiter control frame 608shown in FIG. 6A is a limiter control frame when the brightness is high,and thus in the naked eye view equivalent display mode as well, thelimiter control frame when the brightness is high is used in common inthe two recording modes.

FIG. 7C shows control on the low color temperature side of the limitercontrol frame according to the brightness in the naked eye viewequivalent display mode. FIG. 7C shows an example in which, when thebrightness is high, the color temperature is set to 5000 K, and thecolor temperature is gradually reduced to, for example, 4500 K as thebrightness decreases. 5000 K shown in FIG. 7C is a color temperaturethat corresponds to the low color temperature side of the limitercontrol frame 608 when the brightness is high, which was shown in FIG.6A. Also, 4500 K shown in FIG. 7C is a color temperature thatcorresponds to the low color temperature side of the limiter controlframe 611 when the brightness is low, which was shown in FIG. 6D. Asdescribed above, in the case where the naked eye view equivalent displaymode is set as the display mode, the amount of the light source colorthat remains after the white balance correction has been performed onthe image is set to be larger than the amount of the light source colorthat remains after the white balance correction has been performed onthe still image in the recording mode.

By controlling the limiter control frame as described above, the lightsource color remains more in the naked eye view equivalent display mode,and it is therefore possible to achieve a display closer to what lookslike with the naked eye as compared with the display in the still imagerecording equivalent display mode.

Hereinafter, additional processing performed in the naked eye viewequivalent display mode will be described. The brightness of the imagedisplayed on the viewfinder can be set to be higher than that whenviewed with the naked eye by setting the ISO sensitivity to be high andthe shutter speed to be slow when capturing the image. Accordingly, theuser can recognize a dark subject that cannot be recognized with thenaked eye by looking at the viewfinder. However, when the limitercontrol in the naked eye view equivalent display mode is performed suchthat the light source color remains under conditions where the displayon the viewfinder is brighter than that when viewed with the naked eye,such as a nighttime scene, a problem arises in that the light sourcecolor appears to remain more than that when viewed with the naked eye.

In view of this, in the present embodiment, the above-described problemis solved by performing processing (limiter control range enlargingprocessing) for enlarging the limiter control range under conditionswhere the brightness of the subject on the viewfinder appears to behigher than that when viewed with the naked eye. FIG. 7D shows anexample of color temperature change control that corresponds to the lowcolor temperature side of the limiter control frame in the naked eyeview equivalent display mode in the case where the limiter control rangeenlarging processing is added. The brightness of the subject thatcorresponds to a point indicated by reference numeral 701 shown in FIG.7D is a brightness at which the image displayed on the viewfinder startsto be brighter than that when viewed with the naked eye. The shutterspeed when capturing the image displayed on the viewfinder cannot be setto be slower than the frame rate of the image displayed on theviewfinder. For this reason, when the frame rate changes, the brightnessof the subject at which the image displayed on the viewfinder can be setto be brighter than that when viewed with the naked eye (or in otherwords, the point 701) also changes. Likewise, when the maximum value ofthe ISO sensitivity of the image displayed on the viewfinder changes,the brightness of the subject at which the image displayed on theviewfinder can be set to be brighter than that when viewed with thenaked eye also changes.

In this case, in the present embodiment, a predetermined brightnessthreshold value (the point of the brightness of the subject (the point701)) for changing the color temperature of the limiter control frame ischanged according to the settings of the digital camera such as the ISOsensitivity of the image displayed on the viewfinder. Then, the CPU 103compares the brightness of the subject included in the image with thethreshold value. In the manner described above, by changing the point ofthe brightness of the subject according to the frame rate of theviewfinder, the maximum ISO sensitivity of the image displayed on theviewfinder, and the like, the limiter control frame can be changed asappropriate irrespective of the conditions such as the frame rate andthe maximum ISO sensitivity when image capturing is performed.

In FIG. 7D, a broken line 702 shows an example in which the frame rateof the viewfinder is reduced relative to that of the point 701, or themaximum ISO sensitivity of the image displayed on the viewfinder ishigher than that of the point 701. FIG. 7D shows only two patternsincluding the point 701 and the broken line 702, but a configuration maybe used in which the pattern is more finely divided according to theconditions such as the frame rate or the maximum ISO sensitivity. Also,in the description given above, an example was described in which thecolor temperature that corresponds to the low color temperature side ofthe limiter control frame is changed. However, with respect to the otherside, under conditions where the brightness of the subject on theviewfinder appears to be higher than that when viewed with the nakedeye, control can be performed so as to enlarge the limiter controlframe.

In short, the relationship of the remaining amount of the light sourcecolor described up to here can be shown in FIG. 8 . FIG. 8 showscombinations of the white balance mode for still image recording and thewhite balance mode for viewfinder displaying in the case where thelimiter processing in the auto white balance mode is performed.

The combinations shown in FIG. 8 will be described sequentially from thetop row. First, when the still image recording mode is set to“atmosphere priority mode” and the viewfinder display mode is set to“naked eye view equivalent display mode”, the light source color remainsmore in the viewfinder display mode than in the still image recordingmode. That is, when the naked eye view equivalent display mode (a firstdisplay mode) is set, the amount of the light source color that remainsafter the white balance correction in the naked eye view equivalentdisplay mode is larger than the amount of the light source color thatremains after the white balance correction in the recording mode.

As the second combination, when the still image recording mode is set to“atmosphere priority mode”, and the viewfinder display mode is set to“still image recording equivalent display mode”, the amount of the lightsource color that remains after the white balance correction is the samein the viewfinder display mode and the still image recording mode. Thatis, when the still image recording equivalent display mode (a seconddisplay mode) is set, the amount of the light source color that remainsafter the white balance correction in the still image recordingequivalent display mode is equal to the amount of the light source colorthat remains after the white balance correction in the recording mode.

As the third combination, when the still image recording mode is set to“white priority mode”, and the viewfinder display mode is set to “nakedeye view equivalent display mode”, the light source color remains morein the viewfinder display mode than in the still image recording mode.Furthermore, as the fourth combination, when the still image recordingmode is set to “white priority mode”, and the viewfinder display mode isset to “still image recording equivalent display mode”, the remainingamount of the light source color is the same in the viewfinder displaymode and the still image recording mode.

In the four combinations described above, the remaining amount of thelight source color is larger in “atmosphere priority mode” of therecording mode than in “white priority mode” of the recording mode. Forthis reason, the amount of the light source color that remains after thewhite balance correction is larger when the combination of the naked eyeview equivalent display mode and the atmosphere priority mode is setthan that when the combination of the naked eye view equivalent displaymode and the white priority mode is set. Also, the amount of the lightsource color that remains after the white balance correction is largerwhen the combination of the still image recording equivalent displaymode and the atmosphere priority mode is set than that when thecombination of the still image recording equivalent display mode and thewhite priority mode is set.

Through the processing described above, the white balance correctionvalue calculation processing in S306 ends. The white balance correctionvalue calculation processing for the still image performed in S312 isthe same as that of control performed in the white priority mode or theatmosphere priority mode of the recording mode, and thus a redundantdescription will be omitted.

As described above, the present embodiment is configured to be capableof setting a white balance recording mode that is applied to record theimage as a still image and a white balance display mode that is appliedto display the image before capturing as the still image on the displayunit. When “naked eye view equivalent display mode” of the display modeis set, the amount of the light source color that remains after thewhite balance correction in the image before being captured as a stillimage is set to be larger than the that in the still image in therecording mode. In this way, the image displayed on the EVF can bebrought closer to the colors viewed with the naked eye while keeping thecolors of the still image that is recorded to appropriate colors suchthat the color tone of the light source does not remain excessively. Inother words, even when the color tone of the light source in the stillimage that is recorded is reduced, it is possible to bring the colortone of the light source in the image that is displayed on the EVF closeto the color tone of what looks like with the naked eye.

Second Embodiment

Next, a second embodiment will be described. The first embodiment isconfigured such that white balance correction values are calculatedbased on, out of pixel information regarding divided blocks of acaptured image, pixel information regarding some of the divided blocksof the captured image that are included in a white determining region,and the white balance correction values are corrected according to thedisplay mode. On the other hand, the second embodiment is configuredsuch that the white balance correction values are corrected using atable in which white balance correction values are arranged according tothe light source. The structural elements of an image processingapparatus and white balance correction value calculation processingaccording to the present embodiment are different from those of thefirst embodiment, but other structural elements and processingoperations are the same or substantially the same as those of the firstembodiment. For this reason, structural elements and processingoperations that are the same or substantially the same are given thesame reference numerals, and a description thereof will be omitted, andonly differences will be mainly described.

Configuration of Image Processing Apparatus

FIG. 9 shows an example of a configuration of a white balance controlunit 900 according to the second embodiment. The white balance controlunit 900 is included in the image processing apparatus 106, and performswhite balance processing. The white balance control unit 900 may be acircuit or a software module that performs auto white balancecomputation. The white balance control unit 900 includes a blockdividing unit 901, a light source determining unit 902, and a whitebalance correction value calculation unit 903. The processing performedin each block will be described later.

Series of Operations of Image Recording by Image Capturing

Next, a series of operations of image recording by image capturing willbe described with reference to FIGS. 3 and 10 . The processingoperations are implemented by the CPU 103 deploying a program stored inthe secondary storage device 108 into the primary storage device 104 andexecuting the program, and causing the imaging element 102, the imageprocessing apparatus 106, and the like (including internal structuralelements) to operate.

The CPU 103 performs the processing operations of S301 to S314 shown inFIG. 3 in the same manner as in the first embodiment. In the presentembodiment, the white balance correction value calculation processingperformed in S306 and S312 is different from that of the firstembodiment. Accordingly, this processing will be described withreference to FIG. 10 . In the second embodiment, the secondary storagedevice 108 stores white balance correction values according to variouslight sources are stored in the form of a table. In the table, Rgain,Ggain, and Bgain that represent white balance correction values (thegains applied to R, G, and B signals) that correspond to the whitepriority mode, the atmosphere priority mode, and the naked eye viewequivalent display mode are stored according to the light source.

Furthermore, for the naked eye view equivalent display mode of thedisplay mode, two types of white balance correction values are stored:white balance correction values used when the brightness of the imagedisplayed on the viewfinder is higher than that when viewed with thenaked eye and white balance correction values used when the brightnessof the image displayed on the viewfinder is the same as that when viewedwith the naked eye. At this time, the white balance correction values ofthe light sources are set such that the remaining amount of the lightsource color increases in the following order: the naked eye viewequivalent display mode, the atmosphere priority mode, and the whitepriority mode. Also, in the naked eye view equivalent display mode, thewhite balance correction values used when the brightness of the imagedisplayed on the viewfinder is the same as that when viewed with thenaked eye are set such that the amount of the light source color remainsmore than those used when the brightness of the image displayed on theviewfinder is higher than that when viewed with the naked eye.

In S1001, as in the first embodiment, the CPU 103 divides, using theblock dividing unit 901, the captured image into n by m regions, where nrepresents the number of regions in the horizontal direction, and mrepresents the number of regions in the vertical direction. n and m maybe any integers. In the present embodiment, both n and m are 16, and theimage is divided into a total of 256 regions.

In S1002, the CPU 103 performs light source determining processing usingthe light source determining unit 902. The light source determiningprocessing is performed using the color distribution of block divisiondata created in S1001 and information regarding the brightness of thesubject calculated in S302. The light source determining processing isperformed by selecting, based on the color distribution of the blockdivision data and the information regarding the brightness of thesubject, a light source with the closest features from among the colordistribution of the block division data and the information regardingthe brightness of the subject that corresponds to various light sourcesstored in the secondary storage device 108. The light source determiningprocessing may be configured to perform rule-based processing ordetermine an optimal light source for the color distribution of theblock division data and the brightness of the subject through deeplearning processing using a trained neural network.

In S1003, the CPU 103 calculates white balance correction values for thelight source detected in S1002 based on the table that is stored in thesecondary storage device 108 and in which white balance correctionvalues are arranged according to the light source, using the whitebalance correction value calculation unit 903.

When the still image recording equivalent display mode of the displaymode is selected by the user, the white balance correction valuecalculation unit 903 first determines whether the recording mode is setto the white priority mode or the atmosphere priority mode. Then, whitebalance correction values that correspond to the determined recordingmode are looked up from the table that is stored in the secondarystorage device 108 and in which white balance correction values arearranged according to the light source.

On the other hand, when the naked eye view equivalent display mode ofthe display mode is selected by the user, the white balance correctionvalue calculation unit 903 first determines whether the brightness ofthe image displayed on the viewfinder is the same as that when viewedwith the naked eye or the brightness of the image displayed on theviewfinder is higher than that when viewed with the naked eye. Then,white balance correction values that correspond to the result ofdetermination are looked up from the table that is stored in thesecondary storage device 108 and in which white balance correctionvalues are arranged according to the light source.

In the second embodiment as well, the remaining amount of the lightsource color in each of the combinations of the white balance mode forstill image recording and the white balance mode for viewfinderdisplaying shown in FIG. 8 is the same as that of the first embodiment.When the CPU 103 finishes the processing in S1003, the CPU 103 ends thewhite balance correction value calculation processing in S306, andreturns to the original processing. The white balance correction valuecalculation processing for the still image performed in S312 is the sameas that performed when the image is recorded.

As described above, in the present embodiment, white balance correctionvalues according to various light sources are stored in the secondarystorage device 108 in the form of a table, and white balance correctionvalues that correspond to a light source determined for each block inthe image are applied. In this way, as in the first embodiment, theimage displayed on the EVF can be brought closer to the colors viewedwith the naked eye while keeping the colors of the still image that isrecorded to appropriate colors such that the color tone of the lightsource does not remain excessively. That is, it is possible to bring thecolor tone of the light source in the image that is displayed on the EVFclose to the color tone of what looks like with the naked eye even whenthe color tone of the light source in the still image that is recordedis reduced.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2021-035634, filed Mar. 5, 2021, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing apparatus comprising: one ormore processors; and a memory storing instructions which, when theinstructions are executed by the one or more processors, cause the imageprocessing apparatus to function as: an obtaining unit configured toobtain an image captured by an image capturing unit; a setting unitconfigured to be capable of setting a white balance recording mode thatis applied to record the image as a still image and a white balancedisplay mode that is applied to display the image before being capturedas the still image on a display unit; and a control unit configured tocontrol white balance correction performed on the image captured by theimage capturing unit, wherein, in a case where a first display mode thatis included in the display mode is set, the control unit sets an amountof a light source color that remains after the white balance correctionis performed on the image before being captured as the still image inthe first display mode to be larger than the amount of the light sourcecolor that remains after the white balance correction is performed onthe still image in the recording mode.
 2. The image processing apparatusaccording to claim 1, wherein the display mode further includes a seconddisplay mode, and in a case where the second display mode of the displaymode is set, the control unit sets the amount of the light source colorthat remains after the white balance correction is performed on theimage before being captured as the still image in the second displaymode to be equal to the amount of the light source color that remainsafter the white balance correction is performed on the still image inthe recording mode.
 3. The image processing apparatus according to claim2, wherein, in the white balance correction applied when recording thestill image, the recording mode includes: a first recording mode inwhich the amount of the light source color in the image captured by theimage capturing unit is reduced; and a second recording mode in whichthe amount of the light source color that remains is set to be largerthan that in the first recording mode, and the control unit sets theamount of the light source color that remains after the white balancecorrection is performed on the image before being captured as the stillimage in a case where the second display mode and the second recordingmode are set to be larger than the amount of the light source color thatremains after the white balance correction is performed on the stillimage in a case where the second display mode and the first recordingmode are set.
 4. The image processing apparatus according to claim 1,wherein, in the white balance correction applied when recording thestill image, the recording mode includes: a first recording mode inwhich the amount of the light source color in the image captured by theimage capturing unit is reduced; and a second recording mode in whichthe amount of the light source color that remains is set to be largerthan that in the first recording mode.
 5. The image processing apparatusaccording to claim 1, wherein, in the white balance correction appliedwhen recording the still image, the recording mode includes: a firstrecording mode in which the amount of the light source color in theimage captured by the image capturing unit is reduced; and a secondrecording mode in which the amount of the light source color thatremains is set to be larger than that in the first recording mode, andthe control unit sets the amount of the light source color that remainsafter the white balance correction is performed on the image beforebeing captured as the still image in a case where the first display modeand the second recording mode are set to be larger than the amount ofthe light source color that remains after the white balance correctionis performed on the still image in a case where the first display modeand the first recording mode are set.
 6. The image processing apparatusaccording to claim 1, wherein, in a case where the first display mode isset and the image before being captured as the still image has abrightness lower than a threshold value, the control unit sets theamount of the light source color that remains after the white balancecorrection is performed on the image before being captured as the stillimage in the first display mode to be larger than the amount of thelight source color that remains after the white balance correction isperformed on the still image in the recording mode.
 7. The imageprocessing apparatus according to claim 1, wherein, in a case where theimage before being captured as the still image has a brightness lowerthan a predetermined brightness threshold value in the first displaymode, the control unit sets the amount of the light source color thatremains after the white balance correction is performed on the imagebefore being captured as the still image to be smaller than the amountof the light source color that remains after the white balancecorrection is performed on the still image in a case where the image hasa brightness higher than the predetermined brightness threshold value.8. The image processing apparatus according to claim 7, wherein thecontrol unit determines the brightness of the image before beingcaptured as the still image by comparing a brightness of a subjectincluded in the image before being captured as the still image with thepredetermined brightness threshold value.
 9. The image processingapparatus according to claim 7, wherein the control unit changes thepredetermined brightness threshold value according to settings of thedisplay unit or the image capturing unit.
 10. The image processingapparatus according to claim 9, wherein the control unit changes thepredetermined brightness threshold value according to at least eitherone of a frame rate for displaying on the display unit and an ISOsensitivity set for the image capturing unit.
 11. A control method of animage processing apparatus comprising: obtaining an image captured by animage capturing unit; setting a white balance recording mode that isapplied to record the image as a still image and a white balance displaymode that is applied to display the image before being captured as thestill image on a display unit; and controlling white balance correctionperformed on the image captured by the image capturing unit, wherein, inthe controlling, in a case where a first display mode that is includedin the display mode is set, an amount of a light source color thatremains after the white balance correction is performed on the imagebefore being captured as the still image in the first display mode isset to be larger than the amount of the light source color that remainsafter the white balance correction is performed on the still image inthe recording mode.
 12. A non-transitory computer-readable storagemedium comprising instructions for performing a control method of animage processing apparatus, the control method comprising: obtaining animage captured by an image capturing unit; setting a white balancerecording mode that is applied to record the image as a still image anda white balance display mode that is applied to display the image beforebeing captured as the still image on a display unit; and controllingwhite balance correction performed on the image captured by the imagecapturing unit, wherein, in the controlling, in a case where a firstdisplay mode that is included in the display mode is set, an amount of alight source color that remains after the white balance correction isperformed on the image before being captured as the still image in thefirst display mode is set to be larger than the amount of the lightsource color that remains after the white balance correction isperformed on the still image in the recording mode.